Papers with Visually Rich Document Understanding
MarkupLM: Pre-training of Text and Markup Language for Visually Rich Document Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Existing layout-based pre-training approaches are not easy to apply to VRDU tasks. |
| Approach: | They propose to use markup languages as the backbone for document understanding tasks where text and markup information are jointly pre-trained. |
| Outcome: | The proposed model outperforms existing models on document understanding tasks. |
A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends (2026.findings-acl)
Copied to clipboard
Yihao Ding, Siwen Luo, Yue Dai, Yanbei Jiang, Zechuan Li, Qiang Sun, Geoffrey Martin, Wei Liu, Yifan Peng
| Challenge: | Visually Rich Document Understanding (VRDU) frameworks are a key area of research . early approaches to VRDU relied on manually crafted rules and domain-specific heuristics . conventional deep learning approaches do not integrate the diverse modalities in documents . |
| Approach: | They review recent advances in MLLM-based Visually Rich Document Understanding (VRDU) their findings highlight emerging trends and promising research directions . |
| Outcome: | The proposed frameworks are scalable, reliable, and adaptable, the authors argue . their findings highlight emerging trends and promising research directions . |
ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction (2026.findings-acl)
Copied to clipboard
Zichun Guo, Yuling Shi, Wenhao Zeng, Chao Hu, Haotian Lin, Terry Yue Zhuo, Jiawei Chen, Xiaodong Gu, Wenping Ma
| Challenge: | Empirical evaluations on state-of-the-art MLLMs reveal a significant performance gap . ML models lack the fine-grained cross-modal reasoning required to bridge visual discontinuities. |
| Approach: | They propose a benchmark that renders fragmented documents directly from Markdown to facilitate evaluation of VRDU tasks. |
| Outcome: | The proposed benchmark renders fragmented documents directly from Markdown. |